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ABSTRACT 



A system has an information display and a camera for 
visually observing an observer of the information display. 
The information display and camera may be part of personal 
computer equipped for video conferences. The observer 
controls movement of a pointer on the information display 
by changing positions. The pointer may be moved by 
corresponding rotation or movement of visually identifiable 
characteristic, such as a facial feature of the observer. The 
movement of the pointer may be independent of the area of 
the information display being viewed by the observer. 
Furthermore, the observer may control the information on 
the display by modifying a changeable visual characteristic. 
A closure of a left eye causes generation of a first control 
signal while closure of a right eye causes generation of a 
second control signal. 

26 Claims, 5 Drawing Sheets 
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Vision pointer method and detailed description of the 

apparatus invention 

FIELD OF THE INVENTION FIG. 1 shows an observer and a vision pointer system 

. having an information display and a video camera. The 

This invention generally relates to the area of human input observer or computer user 90 has a face with a profile 100. 

devices for computers and particularly to a vision system for j^e observer's face has a noise having a tip 105, a right eye 

control of a display pointer or cursor. 110> a i c f t eye 120 and a mouth 130. The observer is 

BACKGROUND OF THE INVENTION ™ area 14 f °f S { f ™ a *> n V* °? U P" 

1Q to a persona] computer 160. Area 145 is indicated by dashed 

Most popular personal computers have a pointing device lines from the eyes 110 and 120 of the user 90 to area 145 
as an integral part of their operating system. Such personal of the information display 150. Video camera 170 is shown 
computers include those using Microsoft's Windows- 95 or by dashed lines to be capturing an image of observer 90. 
Apple's Macintosh operating systems. Various types of Video camera 170 captures images of the observer in order 
pointing devices are in use, and include devices commonly 15 to move a pointer on the information display 150, the pointer 
referred to a either a "mouse*', "track pad*' or "track ball". being similar in use to a pointer on a computer monitor 
Such devices are described in U.S. Pat. No. 3,987,685 Oct. which is moved by a conventional mouse, track ball or track 
26, 1976 to Opocensky entitled "Cursor position device" pad. Video camera 170 may further be used for video 
and U.S. Pat. No. 5,424,756 Jun. 13, 1995 to Ho et al. entitle conferencing in a manner known in the art. 
"Track pad cursor positioning device and method" and 2Q FIG. 2 shows a block diagram of the vision pointer system 
hereby incorporated by reference. Pointing devices are used operating in accordance with the present invention. Infor- 
to move a pointer or cursor across an information display, mation display 150 receives signals from the personal com- 
such as a computer monitor, in response to movements by p U ter 160 in order to present images on the information 
the observer of the information display, Aproblem with such display. The images generated by the personal computer 160 
pointing devices is that they require use of a user's hands to 25 include displays associated with application programs such 
operate. Often a user cannot handle a key board associated • as wor d processors, spread sheets, video games, internet 
with the personal computer while operating the pointing p ageSj digital pictures and video conferences. The personal 
device. Thus, what is needed is a pointing device which computer 160 also receives image signals from video cam- 
operates independently of the hands of the user. era 170. The image signals are processed by video signal 

Video conferencing as becoming a popular application for 30 processor 172 to present the image information to computer 

personal computers. More powerful microprocessors, system 175 for applications such as video conferencing, 

improved video compression applications, improved Computer system 175 includes a microprocessor such as a 

modem data rates and lower price video cameras have made Intel Pentium or a Motorola PowerPC microprocessor, as 

video conferencing a more viable option for users of per- well a memory, modem, local area network interface, power 

sonal computers. Such a system is described in U.S. Pat. No. 35 supply, circuits and software including operating system and 

5,351,076, Sep. 27, 1994 to Hata et al., entitled "Terminal application software to process the video signals from video 

system having video telephone equipment cooperating with signal generator 172. Combined with video signal processor 

computer system", which is hereby incorporated by refer- 172, the computer system 175 can be used as a video 

ence. As a result, many personal computers have a video communication means for enabling the user to conduct a 

camera to enable video conferencing. The video camera is 40 video conference using the video camera 170. 

typicaUy directed at the user of the personal computer when xh e i mage signals from video camera 170 are also pre- 

the user is engaged in a video conference. The video camera sented to pattern recognizer 180 which processes the image 

remains directed at the user when the user is observing the signals. Pattern recognizer 180 operates as a position deter- 

information screen and using the personal computer for mining means to determine a position of the observer in 

other computer related applications such a as word 45 response to a visual image of the user and produce a position 

processor, spread sheet, computer game or internet naviga- signal in response thereto. Pattern recognizers capable of 

tion. Thus, it is desirable is to provide an additional appli- determining a position of a user from an image of the user 

cation for the video camera while the user is observing the are know. Descriptions of pattern recognizer may be found 

computer's information display. * m at least the following U.S. patents, which are hereby 

BRIEF DESCRIPTION OF THE DRAWINGS » ^t!?^^ 30, 1991 to La.be*, entitled 

FIG. 1 shows an observer and a vision pointer system "Autonomous face recognition machine"; 

having an information display and a video camera. U.S. Pat. No. 5,561,718 Oct. 1, 1996 to Trew et al., entitled 

FIG . 2 shows a block diagram of the vision pointer system "Classifying faces"; 

operating in accordance with the present invention. 55 u s - Pat - No - 5,629,752 May 13, 1997 to Kinjo, entitled 

FIG. 3 through FIG. 7 show images of the head of the "^ ethod of det ™ n f an ex P°f re amount ^ °P ti " 

observer in various relative positions. T1 f l ™^??*^f?* i}i T? \ nn, n , , 

„ „ , *■ *u u ■ • i U.S. Pat. No. 5,680,481 Oct. 21, 1997 to Prasad et al 

FIG. 8 shows an image of the observer in an initial .... , „ r . , f ' . „ tU A A 

b entitled "Facial feature extraction method and apparatus 

position. 6Q a ncura i ne t W ork acoustic and visual speech"; 

FIG. 9 through FIG. 12 show images of the position of the U<S . Pat. No. 5,715,325 Feb. 3, 1998 to Bang et al., entitled 

observer in various positions relative to the initial position of "Apparatus and method for detecting a face in a video 

FIG * 8 - image"; and 

FIG. 13 through FIG. 16 show changeable visual charac- xj.S. Pat. No. 5,719,951 Feb. 17, 1998 to Shackleton et al., 

teristics of the user. 65 entitled "Normalized image feature processing". 

FIG. 17 shows a flowchart of a vision pointer method Numerous methods of identifying recognizable user char- 
operating in accordance with the present invention. acteristics may be employed in determining the position of 
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the user while remaining within the scope of this invention. 
The position of the pupils of the user's eyes may be used to 
determine an area on the information display at which the 
user is looking, such as area 145 of FIG. 1, and the pointer 
placed in that determined area. Preferably, the relative 5 
position of the user's head is used to determine the position 
of the pointer. For example, if the position signal is gener- 
ated in response to the user's head rotating to the right then 
the pointer signal would cause the pointer to correspond- 
ingly move to the right. If the head of the user is not rotated, 10 
this embodiment has the advantage of allowing the user's 
head to move up or down or side to side without moving the 
pointer. In another preferred embodiment, the position of an 
identifiable visual characteristic of the user may be used to 
generate the position signal and correspondingly move the 15 
pointer. For example, a change in position of the tip of the 
user's nose may be used to generate the position signal and 
move the pointer correspondingly. This embodiment has the 
advantage of enabling pointer movement in response to 
either rotation of the user's head or movement of the user's 20 
head up or down or side to side. Alternately, the position of 
the user's head may be determined relative to a stationary 
background. If the user's head moves to side to side and/or 
up and down relative to the background, then the pointer 
correspondingly moves to the side to side and/or up and 25 
down respectively. 

All but the first of the aforementioned methods have the 
advantage of allowing the user to move the pointer on the 
information display independent of the area of the informa- 
tion display being observed by the user. These embodiments 30 
have the advantage of functionally corresponding to the 
operation of a conventional mouse who's movements are 
also independent of their area of the information display 
being viewed by the user. Thus, a user may view the 
information display without any distraction of a pointer 35 
constantly appearing in the area being viewed. Further, a 
location on the information display may be viewed, such a 
portion of a phrase where a word should be inserted, and 
then the user changes position, by head rotation or 
movement, to position the pointer at the desired insertion 40 
point. A further advantage is that movement of the user's 
hands is not required to place the pointer. 

The position signal from pattern recognizer 180 is applied 
to pointer signal generator 182 which interprets the position 
signal to generate a pointer signal. The pointer signal may be 45 
proportionally related to the position signal or may have 
non-linear components. For example, if a user's position 
changes by a first delta, the pointer signal may cause the 
pointer to move a second delta proportional to the first delta, 
and independent of the time taken to produce the first delta. 50 
The amount of the proportional change may be adjusted by 
the user, typically with a control panel setting. Alternately, 
in a non-linear mode the first delta could produce a third 
pointer delta less than a fourth pointer delta in response to 
the first delta occurring in more time for the third pointer 55 
delta than for the fourth pointer delta. Such linear and 
non-linear pointer movements are known in existing mouse, 
track ball and track pad applications. 

Pattern recognizer 180 also processes the image of the 
observer to recognize changeable visual characteristics of 60 
the user and includes the information in the position signal. 
An example of a changeable visual characteristic of the user 
is the state of the eyes 110 and 120 of the user, which can 
be recognized as either open or closed. The pointer signal 
generator would inhibit movement of the pointer if the 65 
changeable visual characteristic of the user was in a prede- 
termined state. For example, the pointer would not move if 
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both eyes of the user were closed and would move if either 
or both eyes of the user were open. Recognition of the 
changeable visual characteristics gives the user selective 
control of enabling and disabling pointer movement in 
response' to the position of the user. Inhibiting pointer 
movement when both eyes of the user are closed has the 
further advantage of inhibiting pointer movement when the 
user is not looking at the information display, eliminating 
undesirable pointer movements. Additional pointer move- 
ments can be inhibited if it is determined that the user is not 
looking at the display by determining the position of the 
user's pupils. Furthermore, pointer movements can be inhib- 
ited by determining if the distance between the user and the 
information display is greater than a predetermined amount. 

The pattern recognizer 180 also supplies the position 
signal to control signal generator 184. Control signal gen- 
erator generates a control signal related to the pointer signal 
which may function similar to the left mouse and right 
mouse buttons found on a mouse interfacing to the Windows 
95 operating system. These control signals are known to 
personal computer users and may cause selection of text 
identified by the pointer, or enable graphical functions of the 
pointer. The control signal generator monitors the informa- 
tion in the position signal relating to the changeable visual 
characteristic of the user to generate the control signal. 
Preferably, a control signal is generated in response to either 
eye being closed. A left mouse button signal is generated in 
response to the left eye of the user being closed and a right 
mouse button signal is generated in response to the right eye 
of the user being closed. This configuration enables the user 
to continue observing the information screen while gener- 
ating control signals, the user's hands remain free. 
Alternately, other changeable visual characteristics of the 
user may be used, such an open or closed state of the mouth 
or a facial expression. Further, a position of the user may be 
used to generate the control signal such as a tilt of the head. 
For example, a tilt of the head to the left would activate a left 
mouse control signal while a tilt of the head to the right 
would generate the right mouse control signal. The contem- 
plated invention does not preclude the use of hand actuated 
buttons to perform aforementioned control signal functions 
in conjunction with the vision pointer. 

FIG. 3 through FIG. 7 show images of the head of the 
observer in various relative positions. The axis labeled "X" 
and "Y" provide reference coordinates relative to the profile 
100 of the observer and are independent of side to side and 
up/down movement of the observer. FIG. 3 shows the 
position of the head of the user at an initial referenced 
position as shown by the intersection of the X and Y axis 
located at the tip of the user's nose 105. FIG. 4 shows the 
position of the head of the observer rotated in an upward 
direction relative to FIG. 3. The rotation being indicated by 
the tip of the user's nose 105 being located above the 
intersection of the X and Y axis. The rotation of FIG. 4 
causes generation of a pointer signal causing the pointer to 
move up. FIG. 5 shows the position of the head of the 
observer rotated to the observer's right relative to FIG. 3, the 
rotation being indicated by the tip of the user's nose 105 
being located to the observer's right of the intersection of the 
X and Y axis. The rotation of FIG. 5 causes generation of a 
pointer signal causing the pointer to move to the observer's 
right. FIG. 6 shows the position of the head of the observer 
rotated in a left direction relative to FIG. 3, the rotation 
being indicated by the tip of the user's nose 105 being 
located to the observer's left of the intersection of the X and 
Y axis. The rotation of FIG. 6 causes generation of a pointer 
signal causing the pointer to move to the left. FIG. 7 shows 
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the position of the head of the observer rotated in a down- and down pointer movements would correspond to FIG. 5, 
ward direction relative to FIG. 3, the rotation being indicated FIG. 6, and FIG. 7 respectively. A similar absolute position- 
by the tip of the user's nose 105 being located below the ing can be done with FIG. 8 through FIG. 12 where the 
intersection of the X and Y axis. The rotation of FIG. 7 intersection of axis A and axis B are an absolute location 
causes generation of a pointer signal causing the pointer to 5 relative to the viewing area of the video camera. Thus, the 
move down. observer position of FIG. 8 would cause the pointer to be in 

In FIG. 3 through FIG. 7, the pattern recognizer is an the center of the information display. While the position of 

image processor which determines a position of the first FIG. 9 would cause the pointer to be at the top center of the 

identifiable visual characteristic, the user's face or profile information display. As the position of the tip of the user's 

100, relative to the position of the second identifiable visual 10 nose moved up from the position of FIG. 8 to the position 

characteristic, the tip of the user's nose 105, and produces of FIG, 9, the pointer would correspondingly move from the 

the position signal in response thereto. The pointer is then center of the information display to the top center of the 

moved side to side or up and down corresponding to the information display. Similar right, left and down pointer 

relative position of the two identifiable visual characteris- movements would correspond to FIG. 10, FIG. 11, and FIG. 

tics. 15 12 respectively. It should also be appreciated that left-right 

FIG. 9 through FIG. 12 show images of the position of the movements or rotations of the proceeding figures can be 

observer in various positions relative to an initial position of combined with up-down movements or rotations of the 

FIG. 8. FIG. 8 shows the position of the head of the user at proceeding figures to enable moving of a pointer to any 

an initial referenced time as shown by the intersection of the location on the information display. 

A and B axis located at the tip of the user's nose 105. FIG. 20 FIG. 13 through FIG. 16 show changeable visual charac- 

9 shows the position of the observer moving in an upward teristics of the user. In FIG. 13 the both eyes 110 and 120 of 

direction relative to FIG. 8, the movement being indicated the observer are open in which case the aforementioned 

by the tip of the user's nose 105 being located above the pointer movements in response to the position of the 

intersection of the A and B axis. The movement of FIG. 9 observer would occur. FIG. 14 shows the user having a 

causes generation of a pointer signal causing the pointer to 25 changeable visual characteristic where both eyes 110 and 

move up. FIG. 10 shows the position of the head of the 120 are closed. In the preferred embodiment, this would 

observer moving to the observer's right relative to FIG. 8, inhibit generation of pointer signal, causing the pointer to be 

the movement being indicated by the tip of the user's nose stable even if the position of the user changes. FIG. 15 

105 being located to the observer's right of the intersection shows the user having a changeable visual characteristic 

of the A and B axis. The movement of FIG. 10 causes 30 where the right eye 110 is closed and the left eye 120 is open, 

generation of a pointer signal causing the pointer to move to In the preferred embodiment, the pointer signal would be 

the observer's right. FIG. 11 shows the position of the head generated in response to the position of the user and gen- 

of the observer moving in a left direction relative to FIG. 8, eration of a control signal corresponding to a right mouse 

the movement being indicated by the tip of the observer's button signal, FIG. 16 shows the user having a changeable 

nose 105 being located to the left of the intersection of the 35 visual characteristic where the left eye 120 is closed and the 

A and B axis. The movement of FIG. 11 causes generation right eye 110 is open. In the preferred embodiment, this 

of a pointer signal causing the pointer to move to the would cause the pointer signal would be generated in 

observer's left. FIG. 12 shows the position of the head of the response to the position of the user and generation of a 

observer moving in an downward direction relative to FIG. control signal corresponding to a left mouse button signal. 

8, the movement being indicated by the tip of the user's nose 40 FIG. 13 through FIG. 16 show the operation of a display 

105 being located below the intersection of the A and B axis. control device for controlling information on an information 

The movement of FIG. 12 causes generation of a pointer display observed by a user. The image processor or pattern 

signal causing the pointer to move down. generator 180 coupled to the video camera 170 which 

It should be appreciated that the pointer movements captures an image of the user having at least one changeable 

which result from the observer movements of FIG. 8 through 45 visual characteristic capable of changing between a first 

FIG. 12 would also result from the rotations of FIG. 3 visually identifiable state and at least a second visually 

through FIG. 7, assuming the user is not moving side to side identifiable state. The image processor determines the 

or up and down, because the tip of the observer's nose 105 . changeable visual characteristic as being in either the first or 

correspondingly moves when the head of the user is rotated. second state. The control signal generator 184 is coupled to 

Preferably, pattern recognizer 180 identifies the tip of the 50 the image processor 180 and generates an information 

observer's nose as the identifiable visual characteristic. control signal in response to the first but not the second state. 

However, it should be further noted that the tip of the user's The first state corresponds to one of the two eyes being open 

nose need not be used, other identifiable visual characteris- and the other of the two eyes is closed. The second state 

tics of the observer may be used including the change in corresponds to both eyes being open or both eyes being 

positions of the eyes, mouth or hair or the movement of the 55 closed. The control signal includes a first control signal for 

head of the user relative to a stationary background. controlling a first type of display control, such as a right 

The movement of the pointer FIG. 3 through FIG. 12 has mouse button press, in response to only the right eye being 

been described in terms relative to an initial reference closed and the left eye being open, and the control signal 

position. It should be appreciated that an absolute position further includes a second control signal for controlling a 

method could also be used. For example, the user position of 60 second type of display control different from the first type of 

FIG. 3 would cause the pointer to be in the center of the display control, such as a left mouse button press, in 

information display. While the position of FIG. 4 would response to the left eye being closed and the right eye being 

cause the pointer to be at the top center of the information open. 

display. As the user's head rotated up from the position of FIG. 17 shows a flowchart of a vision pointer method 

FIG. 3 to the position of FIG. 4, the pointer would corre- 65 operating in accordance with the present invention. The 

spondingly move from the center of the information display flowchart of FIG. 17 corresponds to the observer positions 

to the top center of the information display. Similar right, left referenced in the description accompanying FIG. 8 through 
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FIG. 12. Step 200 checks if both eyes are closed, and if not tifiable visual characteristic and for producing the posi- 

proceeds to step 210 to determine the position of the nose of tion signal in response thereto; and 

the user. Tlien a predetermined delay is executed at step 220 ft pointer si { generator for prov i dill g a po i nt er signal 

which is preferably 0.1 seconds Step 230 determines the for movi a inter QQ ^ informatioil display in 

current position of the nose using the aforementioned pattern s response to the position signal. 

recognition. Step 240 compares the current position of the 2 ^ imin device according to claim t wherein the 

nose with the prior position of the nose. Step 250 determines uscr farthfir has at kast Qne changeablc visual charact cristic 

the position signal to be the difference between the current capable of cnanging between a first visually identifiable state 

position of the nose and the prior position of the nose. Step and at ^ a second identifiable state included in 

260 converts the position signal into the pointer control io the visual image wherein 

signal as previously described, and the pointer is moved at . . 

* >n_ . . f ,u • j 4 - a • 4. *»on sa id image processor determines the changeable visual 

step 270. The status of the eyes is determined m step 280 , T • t. • • . ^ . A A 

. „ ** j ** *+• inn characteristic as being in either the first or second state, 

using the aforementioned pattern recognition. Step 290 , c , , f L . . A , . 

, , . r , , f j. f , *. - « t -inn and lurther wherein the position signal is generated in 

checks if both eyes are closed, and if true returns to step 200. , , 4 . . r c f 4 b ,. ..... . 

. i nn j » j -, An * u 1 -«4U i a response to determination of the first state and inhibited 

Otherwise step 290 proceeds to step 300 to check if the left 15 . r . r .1 * . . 

1 1 *r * i ha « * 1 • 1 m response to determination of the second state, 

eye is closed, if true step 310 generates a control signal * . j ■ j- . 1 ■ L ■ .1. 

. . . 4 u , fi u 44 • 10* u 1 3. The pointing device according to claim 2 wherein the 

equivalent to the left mouse button signal. Step 320 checks .,1. i_i ?i • j 1 j 

•ca. • Li • 1 1 1 *c * 4 nn 4 user nas at least one eye capable of being opened or closed 

if the nght eye is closed, and if true step 330 generates a , . . , . / . y L * • j ♦ .l 

... : 1 . , ,1 . r , „ . . and the changeable visual characteristic corresponds to the 

control signal equivalent to the nght mouse button signal. , 1 & , „ , 4 , , 4 r , , , , 

™ & ° A , ,1 t « t fun* -1 if * at least one eye. and the first state corresponds to the at least 

Thereafter the method returns to step 220 to wait and begin 20 . . j j j K . j . 

r-* * j * 1 • 1 one eye bemg opened and the second state corresponds to 

another sequence of pointer movement and control signal ' A & r , . , , r 

a . _ r ^ 4 the at least one eye being closed, 

generation. The now chart of FIG. 17 preferably operates . . . lL 

& , . t . „ ...f . r 4. The pointing device according to claim 2 wherein the 

substantially in software operating within the microproces- , f & , li r i_ • j 

- 4t _ , * « j . . * * user has two eyes, each eye capable 01 being opened or 

sor of the personal computer. The determinations of steps T , J4l _ . 1 , • , . . • ^ 

nn -nn j ion _r j u *u j- j closed, and the changeable visual characteristic corresponds 

210, 230 and 280 are performed by the aforementioned 25 t tU ! , 7, £ . 4 . j * * 1 * 

n T-tT/-i . A1 *i j* - - to the two eyes, and the first state corresponds to at least one 

pattern recognizer 180 of FIG. 2. Alternately, determining r* u * u • ^ j *'* 

f, r lL u • i £ j u 1 ■ of the two eyes being opened and the second state corre- 

the position , o ! the nose can be amplified by placing an ^ ^ ^ aQd ^ a third 

easily identifiable marker, such as a blue dot, on the nose f . / 4 . . , , 4 . tL 

*_ . .j V.- . , ™ state corresponds to one eye bemg opened and the other eye 

which can be readily identified by a video processor. The closed and 

identifiable marker could also be placed elsewhere to enable 30 * 

determination of the position of the observer, such as a said ima S e Pressor further determines the state of the 

marker incorporated into glasses or other article worn by the changeable visual characteristic and the pointing 

user device further comprises 

It should be further appreciated that for computers with a a control signal generator for generating a control signal 

single mouse buttons, such as the Macintosh, either eye 35 in response to the third state for controlling information 

being closed could be used to generate a control signal on the information display. 

corresponding to the single mouse button signal. Also, one 5. The pointing device according to claim 1 wherein the 

of the control signals could correspond to another function user has a face having a nose and the first identifiable visual 

commonly associated with a computer key board such as a characteristic corresponds to the face and the second iden- 

back space, shift or delete function. 40 tifiable visual characteristic corresponds to the nose wherein 

Thus, what has been provided is a pointing device which said image processor produces the position signal com- 

operates independently of the the hands of the user and an prising 

additional application for the video camera while the user is an up signal when tne nose of the user ^ positioned 

observing the information display. Further, the invention can upward relative to the face, 

be used in conjunction with a voice recognition system to 45 , , , £ . , 

, 4 4 . „ J . it _ . ... a down signal when the nose of the user is positioned 

substantially replace the conventional computer input means downward relative to the face 

of a key board and a mouse. This new combination results . / . . ... 

in a new but familiar user input means. The user would no a left ^ hen iht t nose of the c user 15 Positioned 

longer need to physically touch a computer, the new inter- towards a left Sldc relatlve t0 the facc > and 

face results from vision and sound inputs from the user. 50 a ri e nt signal when the nose of the user is positioned 

However, the familiar functionality of the conventional towards a right side relative to the face, and whereby 

mouse is substantially similar to the functionality of the pointer moves up, down, left, and/or right in 

vision pointer described herein and the familiar functionality response to the position signal having a corresponding 

of the conventional keyboard is substantially similar to the U P» dow n, left and/or right signal, 

functionality of the voice recognition system. 55 6 - ^ P^ting device according to claim 1 wherein 

I claim: said pointer signal moves the cursor to an absolute 

1. A pointing device for pointing to a location on an position on the information display in response to the 

information display observed by a user observing the infor- determined position of the first identifiable character- 

mation display, the pointing device comprising: is tic relative to the position of the second identifiable 

a position determiner for determining a position of the 60 visual characteristic by said image processor, 

user in response to a visual image of the user and for 7. The pointing device according to claim 1 included 

producing a position signal in response thereto, within a computer system, the computer system further 

wherein the user has an at least first and second comprising 

identifiable visual characteristics and said position , voice recognizer for processing voice inputs from tbe user 
determining means comprises an image processor for 65 and adapted to operate in conjunction with said point- 
determining a position of the first identifiable visual ing device to provide a user interface responsive to 
characteristic relative to a position of the second iden- visual and sound inputs from the user. 
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8. A display control device for controlling information on 
an information display observed by a user, the display 
control device comprising: 

an image processor coupled to a video camera for cap- 
turing an image of the user having at least one change- 
able visual characteristic capable of changing between 
a first visually identifiable state and at least a second 
visually identifiable state, said image processor for 
determining the changeable visual characteristic as 
being in either the first or second state; and 

a control signal generator coupled to said image processor 
for generating an information control signal in response 
to the first state, wherein the changeable visual char- 
acteristic is substantially independent of a direction of 
movement of the user and the information control 
signal controls information presented on the informa- 
tion display and further wherein the user has two eyes 
and the changeable visual characteristic corresponds to 
the two eyes, and the first state corresponds to one of 
the two eyes being open and the second state corre- 
sponds to both eves being open or both eves being 
closed. 

9. The display control device according to claim 8 further 
comprising: 

a position determining means coupled to the video camera 
for determining a position of the user in response to the 
image of the user and for producing a position signal in 
response thereto; and 

a pointer signal generator means for providing a pointer 
signal for moving a pointer on the information display 
in response to the position signal. 

10. The display control device according to claim 9 
wherein the image of the user is further capable of a third 
visually identifiable state wherein 

said image processor further determines the visual char- 
acteristic as having the third state and producing the 
information control signal in response thereto; and 

said position determining means is responsive to said 
image processor for inhibiting changes in the position 
signal in response to the third state, 

11. The display control device according to claim 10 
wherein the user has two eyes and the third state of change- 
able visual characteristic corresponds to the two eyes being 
closed. 

12. The display control device according to claim 8 
included within a computer system, the computer system 
further comprising 

voice recognizer for processing voice inputs from the user 
and adapted to operate in conjunction with said display 
control device to provide a user interface responsive to 
visual and sound inputs from the user. 

13. The display control device according to claim 8 
wherein the two eyes of the user include a left eye and a right 
eye and the information control signal includes a first control 
signal in response to only the right eye being closed and the 
left eye being open, and a second control signal different 
from the first control signal in response to the left eye being 
closed and the right eye being open, 

14. A pointing device for pointing to a location on an 
information display observed by a user having at least one 
changeable visual characteristic capable of changing 
between a first visually identifiable state and at least a 
second visually identifiable state, the pointing device com- 
prising: 

a position determiner for determining a position of the 
user in response to a visual image of the user and for 
producing a position signal in response thereto; and 
having 
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an image processor responsive to the visual image for 
producing the position signal and for determining the 
changeable visual characteristic of the visual image as 
being in either the first or second state, wherein the 
position signal is generated in response to determina- 
tion of the first state and inhibited in response to 
determination of the second state; and 

a pointer signal generator means for providing a pointer 
signal for moving a pointer on the information display 
in response to the position signal, wherein the user has 
two eyes, each eye capable of being opened or closed, 
and the changeable visual characteristic corresponds to 
the two eyes, and the first state corresponds to at least 
one of the two eyes being opened and the second state 
corresponds to both eyes being closed, and further 
wherein a third state corresponds to one eye being 
opened and the other eye being closed and 

said image processor further deterrnines the third state of 
the changeable visual characteristic and the pointing 
device further comprises 

a control signal generator for generating a control signal 
in response to the third state for controlling information 
on the information display. 

15. The pointing device according to claim 14 wherein the 
user has an at least first and second identifiable visual 
characteristics and said position determiner comprises 

an image processor for determining a position of the first 
identifiable visual characteristic relative to a position of 
the second identifiable visual characteristic and for 
producing the position signal in response thereto. 

16. The pointing device according to claim 15 wherein the 
user has a face having a nose and the first identifiable visual 
characteristic corresponds to the face and the second iden- 
tifiable visual characteristic corresponds to the nose wherein 

said image processor produces the position signal com- 
prising 

an up signal when the nose of the user is positioned 

upward relative to the face, 
a down signal when the nose of the user is positioned 

downward relative to the face, 
a left signal when the nose of the user is positioned 

towards a left side relative to the face, and 
a right signal when the nose of the user is positioned 

towards a right side relative to the face, and whereby 

the pointer moves up, down, left, and/or right in 

response to the position signal having a corresponding 

up, down, left and/or right signal. 

17. The pointing device according to claim 14 included 
within a computer system, the computer system further 
comprising 

voice recognition means for processing voice inputs from 
the user and adapted to operate in conjunction with said 
pointing device to provide a user interface responsive 
to visual and sound inputs from the user. 

18. A display control device for controlling information 
on an information display observed by a user, the display 
control device comprising: 

an image processor coupled to a video camera for cap- 
turing an image of the user having at least one change- 
able visual characteristic capable of changing between 
a first visually identifiable state and at least a second 
visually identifiable state, said image processor for 
determining the changeable visual characteristic as 
being in either the first or second state; 

a control signal generator coupled to said image processor 
for generating an information control signal in response 
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to the first state, whereby the information control signal 
controls information presented on the information dis- 
play; 

a position determiner coupled to the video camera for 
determining a position of the user in response to the 5 
image of the user and for producing a position signal in 
response thereto; and 

a pointer signal generator means for providing a pointer 
signal for moving a pointer on the information display 
in response to the position signal, wherein the image of 10 
the user is further capable of a third visually identifiable 
state wherein 

said image processor further determines the visual char- 
acteristic as having the third state and producing the 15 
information control signal in response thereto; and said 
position determining means is responsive to said image 
processor for inhibiting changes in the position signal 
in response to the third state. 

19. The display control device according to claim 18 2Q 
wherein the user has two eyes and the third state of change- 
able visual characteristic corresponds to the two eyes being 
closed. 

20. The display control device according to claim 18 
wherein the user has two eyes and the changeable visual 25 
characteristic corresponds to the two eyes, and the first state 
corresponds to one of the two eyes being open and the 
second state corresponds to both eyes being open or both 
eyes being closed. 

21. The display control device according to claim 20 3Q 
wherein the two eyes of the user include a left eye and a right 
eye and the information control signal includes a first control 
signal in response to only the right eye being closed and the 
left eye being open, and a second control signal different 
from the first control signal in response to the left eye being 35 
closed and the right eye being open. 

22. The display control device according to claim 18 
included within a computer system, the computer system 
further comprising 

voice recognizer for processing voice inputs from the user 40 
and adapted to operate in conjunction with said display 
control device to provide a user interface responsive to 
visual and sound inputs from the user. 

23. A pointing device for pointing to a location on an 
information display observed by a user having at least one 45 
changeable visual characteristic capable of changing 
between a first visually identifiable state and at least a 
second visually identifiable state, the pointing device com- 
prising: 

a position determining means for determining a position 50 
of the user in response to a visual image of the user and 
for producing a position signal in response thereto; and 
having 

an image processor responsive to the visual image for 
producing the position signal and for determining the 55 
changeable visual characteristic of the visual image as 
being in either the first or second state, wherein the 
position signal is generated in response to determina- 
tion of the first state and inhibited in response to 
determination of the second state; and 
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a pointer signal generator means for providing a pointer 
signal for moving a pointer on the information display 
in response to the position signal, wherein the user has 
an at least first and second identifiable visual charac- 
teristics and said position determining means com- 
prises 

an image processor for determining a position of the first 
identifiable visual characteristic relative to a position of 
the second identifiable visual characteristic and for 
producing the position signal in response thereto and 
further wherein the user has a face having a nose and 
the first identifiable visual characteristic corresponds to 
the face and the second identifiable visual characteristic 
corresponds to the nose wherein 

said image processor produces the position signal com- 
prising 

an up signal when the nose of the user is positioned 

upward relative to the face, 
a down signal when the nose of the user is positioned 

downward relative to the face, 
a left signal when the nose of the user is positioned 

towards a left side relative to the face, and 
a right signal when the nose of the user is positioned 

towards a right side relative to the face, and whereby 

the pointer moves up, down, left, and/or right in 

response to the position signal having a corresponding 

up, down, left and/or right signal. 

24. The pointing device according to claim 23 wherein the 
user has at least one eye capable of being opened or closed 
and the changeable visual characteristic corresponds to the 
at least one eye, and the first state corresponds to the at least 
one eye being opened and the second state corresponds to 
the at least one eye being closed. 

25. The pointing device according to claim 23 wherein the 
user has two eyes, each eye capable of being opened or 
closed, and the changeable visual characteristic corresponds 
to the two eyes, and the first state corresponds to at least one 
of the two eyes being opened and the second state corre- 
sponds to both eyes being closed, and further wherein a third 
state corresponds to one eye being opened and the other eye 
being closed and 

said image processor further determines the third state of 

the changeable visual characteristic and the pointing 

device further comprises 
a control signal generator for generating a control signal 

in response to the third state for controlling information 

on the information display. 

26. The pointing device according to claim 23 included 
within a computer system, the computer system further 
comprising 

voice recognizer for processing voice inputs from the user 
and adapted to operate in conjunction with said point- 
ing device to provide a user interface responsive to 
visual and sound inputs from the user. 
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